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Virtually all the research on multilingual bilingual education (BE) 1s 
comparative in nature, yet we have not really 'paid any attention to the 
problems which a comparative approach entails. The only reference in the 
literature on bilingual and migrant education which I have come across 
which specifically mentions some df the problems of a comparative analysis 
is Ekstrand's "Migrant Adaptation: A Cross-Cultural Problem" (1978:30) 
and then only in passing. In this paper I would like to explore some of 
the functions and problems of a comparative study of bilingual education 
in order to clarify the question of general liability of BE research. 
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As Simon points outmost empirical research 1n psychology, sociology, 
marketing research, education, anthropology, political science, and all 
other branches of social science except economics is comparison research, 
although sometimes the comparison Is part of research Intended to establish 
cause and effect. (1969:63)." Comparative study takes different forms 
in the various disciplines, basically because of the different problem,, 
formulations. One claim is that the term "comparative approach does not, . 
as has sometimes been claimed, properly designate a specific method In 
social research, but rather a special focus on cross-societal,, institutional, 
or macrosocletal aspects of societies and social analysis" (Shi Is in 
Eisenstadt 1969:423). The methodological problems then are not distinct 
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from those of any other type of sociological research Except as the choice 
of topic in comparative study may necessitate special types of data. 



In sociology, comparative research usually deals with lnypothesis testing 
atbout social behavior and institutions, through statistical techniques, 
over a wide sample of societies in order to find universal trends and 

.general laws. Social anthropologists (in which field the term "compara- 
tive method" first seems to have become established) similarly work for 
"the development of general propositions about culturally regulated human 
behavior" which some believe will lead to the discovery of true sociological 
Haws", similar to the law of gravity. Leach (1969:339) points out that 
this analogy with the natural sciences simply does not work in the study 
of man because in contradistinction to the subject matter of natural, 
science, man has a vfUl of his own. Instead of discovering laws, the pur- 
pose of cross-cultural comparison in social anthropology should be "to 
discover what is humanly feasible rather than to demonstrate what is 
statistically probable. 1 Cross-cultural comparison here becomes a means of 

.understanding the humanity of human beings. It is not a question of demon- 
strating that culture is like nature, but of showing how culture differs 
from nature" (341:342). In psychology, 1 comparison research need not be 
cross-cultural but consists of what research is commonly held to be: exr- 
perimental design andhypothesis testing with pre-specified variables through 



'Comparative psychology proper is concerned wi^h the behavior of different 
species of living organisms, typically animals, and leads to the 'specifi- 
cation of similarities and differences in behavior between species in order 
to relate an animals behavior to its evolutionary background. (Waters and 
Bunnel 1969). 

•■Cross-cultural psychology is a meta-method with all of the areas of psycho- 
logy grist to its mill": (Brislin et. 4J. 1975:7) 



the collection of quantification data from groups which are then com- 
pared. The problems of comparison here are common to all social and be- 
havioral research but when this , type of research design attempts to el uci- 
date questions of bilingual education, there, are problems associated with 
the specific types of data. For example, the crucial necessity to control 
for SES in studies on BE programs is (or should be) common to all research, 
but Cummins' attempt of theory building of L 1 and L 2 acquisition and their 
interdependence necessitates the distinction in the language data between 
CALP (cognitive-academic language proficiency) and BICS (basic interpersonal 
communicative skills), a problem of conceptualization and operational ization 
of data specific to this problem formulation. (Cummins 1982) Bilingual 
education,- certainly not a discipline of its own, draws primarily on 
psychology, anthropology, and sociology in its research and so will share 
the functions and problems of research in those fields. Science does not 
solve problems, but it can help people make better decisions. That I and 
so many other people have stopped smoking is certainly the direct result 
of fhe findings of medical research, to take an example which involves . 
individual decision making. In bilingual education, the research is typi- 
cally marked by a practical and empirical approach i often specifically . 
designed to help people make better decisions, "such as to.fgren's 
Modersmalsklasser eller sanwiansatta klasser for barn tiji invandrare, (1981 ) 
('Mother tongue classes or integrated classes for children of immigrants: 
A theoretical discussion and empirical testing of some selected propositions 
and development of a casual model for immigrant children's success in 
school,' my translation) or Wrede's El§^, j*r£^ 
uDDfattning av arbetet i tre finska llostadieklasser j Eskilstuna kpniTiun 
(1979) ('Students', parents' and teachers' experience and opinion of the , 
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work in three Finnish elementary classes in Eskilstuna township,' my 
translation). The problem is of course that the decisions they advocate 
ar6 diametrically opposed, and I think it is exactly the frequent pheno- 
menon of contradictory data in the research of BE which has led to the 
necessity of a cross-societal, cross-national, cross-cultural approach in 
order to find universal trends and test hypotheses for the schooling of 
children in another language than their mother tongue. I have in mind such 
studies as Ekstrand's Migrant Adaptation : A Cross-Cultural Problem (1978) 
or my own Ethnic Relations and Bilingual Education : Accounting for 
contradictory data (1975). 

Much BE researches evaluation research of specific programs and -so by 
necessity treats the program as the independent variable, but I think most 
of us are by now in agreement that such findings hold only limited general i 
zability and that to understand such findings one has to consider school 
programs or treatments as, intervening or contextual variables and look 
to socio-economic and cultural factors for causal /explanations of language 
acquisition, of school grades, of social mobility, of employment rates, of 
however one chooses to operational ize program success. One of the major 
tasks, then, in the comparative study of Bf Wclmes to identify under 
What conditions , the classical question of sociology, children will 
successfully manage schooling in two languages.- This approach typically 
consists of an analysis of case studies (the evaluation/measurement case 
studies are not only useful here but indispensable) in light of some guiding 
hypotheses. * .. 

At times the theories which form the bases for studies are also examined in 



a comparison of the case studies which serve to document the theories: 
such a study 1s Ekstrand's Early Bilingual ism : Theories and Facts (1979) 
where Ms findings support Cummins' that older students make more efficient 
language learners. Cummins' study 1s interesting in that he deliberately 
compares two groups of students With different background characteristics, " 
namely upper-middle class Japanese and Vietnamese refugee students in 
order to test the general izability of his Interdependence hypothesis ■„•• 
(Cummins 1982). Such studies are rare. Case studies are of course also 
used to develop typoloqies of various kinds, (See Mackev » 1<»7n) . There 1s also 
replication* of case studies, such as the Culver C1tv reDllcation (Camobell 
1972) Of the St. Lambert study (Lambert and Tucker 1972). 
Frequent variables in the experimental design type research are sex, age, 
ethnic identity, sense of self, vocational choice, school grades and high 
school attendance to mention some I culled from the Swedish research, where 
the behavior of migrant students 1s compared with those of Swedish youth. 
Lofgren and Ouvinen-Birgerstam make the Interesting observation that this 
research, i.e. research on migrants, "has to a high degree been characterized 
by fault-finding, i.e. it has been geared to look for deficiencies" (1980:102) 
and go on to question the resul|^|uch research on migrants' , pride and 
Swedes' attitudes.! • 1 . - 



I believe these types of research: the search for social conditions which 
are predictable, theory testing, typology building, replication and the 
studies with experimental design (which are inherently comparative) con- 
stitute the major functions of comparison in B|i In addition, we have , 
Constrastive Analysis in. linguistics and occasionally what amounts to a 
contrastive analysis even if not so labeled in. anthropology, such as 



Freudenthal, Narrowe, and Sachs' Turkar i svensk forort (n.d. ) ( 'Turks 
tn Swedish suburbia'), where it 1s specifically the Turks' unSwedish be- 
havior which 1s singled out for description, also. a form of comparison. 

Eisenstadt points out that the construction of types for purposes of 
comparative analysis poses several methodological questions. One of these 
1s the selection of units comparison. Ekstrand (1981a) rejects the notion 
of a critical period 1n language acquisition while Scovel (1981) breaks \ 
down language into separate skills and so argues for a critical period for 
the acquisition of pronunciation. Their units of comparison are dlf- 
ferent. I have already mentioned that Cummins sees it necessary to 
break down language acquisition Into two units he calls CALP and BICS 
or lately "context-reduced'' and "context-embedded" language proficiency 
(1982) where the very labels for the units contain a concep^al explanation. 
Context-reduced language acquisition favors older learners and so sheds 
further light on the critical period problem. 

We talk glibly about Immigrants and immigrant education but who are they? 
This has caused considerable definitional difficulty for the Swedish 
statisticians (see Re1 nans 1980). We know that the Finns as a group have 
the lowest percentage of students continuing their schooling after gradua- 
tion from- the compulsory grundskolan but who are the Finns? Are the 
Tornedalfinns Finns? Are the Swedef inns Finns? The answer may seem 
perfectly obvious but 1f the question is asked in order to extend the rights 
to mother ^tongue instruction the answer 1s not obvious. The rationale 
for selection of unit of comparison and analysis Hes as always with the 

* ■ • . . * 

research question, with the purpose of the research. 



Another problem is the construction of indices through which the variables 
can be compared. We talk about multicultural education, but what is it,? 
I don't believe that I have ever seen a multicultural education classroom, not 

in my own class room J n Tangier Where we represented eight different 

/ ' ' ••' • , ;. • . •' . fr.'. . - 

cultures 2 , but until we have a generally accepted construction of indices 

of what constitutes multilingual education I really can't telF^ecause I 

have no way of comparing that classroom with, say, my Katrineholm class- 

room along the variables of culture. 

Comparability of variables is a problem in most research buji especially in 
research which uses case studies for secondary data. Comparing, 
as is so often done, the Canadian immersion programs with the monolingual 
schooling in the L 2 of migrant children is a classic error of comparison, 
of comparing the noncomparable.- The Canadian programs show us, in Leach's 
terms, what is humanly feasible, but it remains to spell out under what 
conditions. Comparison research often compares against a known standard, 
be it IQ scores, TOEFL scores or standardized subject matter tests. Much 
of the Swedish research on migrants compares the scores f,or immigrant 
youth, (employment figures, "gymnasium attendance, and test scores) with 
the Finnish or Swedish national norms. But immigrant youth, as both 
Ekstrand (1981b) and Tingbjorn and Andersson (1981) point put, are pri- 
marily members of the working class while the scores are normed on ^he 
entire population so that such comparison will give a skewed result. 
Sometimes faulty comparison is a result of conceptual confusion, such as 

■ *- ": _^ . ■ " ' • *» • * 

* ' ' ' ■"•..-'*.■ ' ' * 

" 2 Arab, Berber, English, Gibraltarian, Indian, Spanish, Swedish, US. 



comparing foreign language learning with second language learning, 
sometimes it is just a lack of facts such as comparing a Swedish grade 9 
with a US grade 9 1n matters of age (Swedes begin school at age 7). I 
know of no formula for achieving comparability except' constant vigilance 
of carefulness and thoughtful ness. 

Comparability is closely tied to the^problem of sampling. In her review 

of the literature on IQ and bilingual ism, Darcy (1963) found a negative 

correlation in many studies. What she does not point out is the faulty . '■■ 

sampling procedures of those studies: most of the subjects were recent 

immigrants, members of the working class, and apparently 1n many cases not 

proficient in English. In cross-national studies, sampling often involves 

a compromise between representativeness of group or country and equivalence 

across countries. Osgood et al . (1975) give the example of Indian college 

students being more highly selected (less representative) than an equivalent 

sample of Dutch college students. In the Lambert and KHrteberg study 

Children's Views of Foreign People : A Cross-National Study (1967), they 

wanted to compare children's images of themselves and foreigners as a 

function of their nationality and age: 

Therefore they wanted both equivalence, in the sense of age 
levels, Sex ratios, intelligence an4 the like, and represen- 
tativeness, In the sense of socio-economic level and the 
like so that differences could not be attributed to educa- 
tion of parents, for example. This required a compromise, : 
which in their case appeared as a form of stratified sam- 
pling within otherwise equivalent groups. (Osgood et al. 
(1975:20) 

Osgood's point is the nature of the compromise depends on the purpose of 
the research. 

^Unfortunately Swedish- for immigrants has become known as Svenska som 
frammande sprak, 'Swedish as..a foreign lalguage' which is not ^helpful. 



Measurement presents a number of problems. One 1s at the heart of all 
research, that of finding empirical proxies to measure for conceptual 
variables, the problem of operat1onal1zat1on. I have argued elsewhere 
(1980) for the need to evaluate the BE programs with Indicators like school 
drop out rates and employment figures 1n addition to standardized test 
scores and won't repeat that argument here. Basically that 1s a problem In 
goal perception. Mostly Jn comparison research, I should think, the opera- 
tional 1zat1 on problem 1s one of validity of the instruments. In many of the 
Darcy studies, the IQ. tests really functioned as language, proficiency 
tests and 1t was not IQ one compared but knowledge Of English. Working 
with case studies, one needs to pay attention to the Instruments use,d. 
Time of treatment fits 1n marginally here. I remember one dissertation which 
attempted to evaluate a bilingual education program after eight hours of 
treatment. That is just plain silly. The Rock Point study (Rosier & Holm 
1980) needed six years to show a positive result. ....... 

Sometimes the problem of operational Ization has to do with finding Indi- 
cators for concepts which are unique to research on bilinguallsm. The 
rationale most often cited for the Swedish policies on mother tongue ' 
teaching is that of halvsorakighet , semi lingual ism or more correctly 
double semi lingual ism, a notion popularized by HansegSrd in 1968 which 
claims that bilingual children may learn neither of their two languages 
well. The press abounds with reference to semi! InguaT ism, the Finnish 
associations c^alm It as a rationale in their demand for monolingual 
Finnish university in Sweden, and university students essays repeatedly 
make comments that "researchers are agreed that semi l ingual ism. . .". 
The fact of the matter 1s that semi lingual ism as a conceptual variable 
has never beeh operational i zed to anyone's satisfaction so researchers 



cannot very well agree on something that may not exist. Actually, they 
don't agree: Lawn's work (1974) is noticeable in this context. The 
counter argument 4 leads into the quantitative/qualitative data discussion. 
Finding an empirical proxy for semilingualism will trivialize the concept, 
goes the claim. If it is a linguistic phenomenon, it ought to be observable, 
say the linguists. (See e.g. Hyl tens tarn & Stroud 1982) At present, there 
is a stand still. -But as a major problem in Swedish research, operationali- 
zation of' halvsprlkighet remains. Before it becomes the major rationale 
for Swedish language policies, the existence of such a lavage phenomenon 
needs to be established, measured and described or the notion should be 
dismissed; 

Qualitative data has other problems in comparison research. Cross-cultural 
anthropologists worry a lot about the general izabiltty of their data, 
mostly for reasons of validity. Would another anthropologist with other 
training and maybe another worldview have seen and described the same 
things, reached the same interpretations? In the immigrant research one 
will occasionally find statements, comments, poems by the immigrants 
themselves which is a form of operational zing attitudes and feeli/jgs. 
When such statements are representative (i.e. reliable) of the group, they ' 
make excellent data, but they are of course esepctally vulnerable .to 
researcher bias in their selection. I don't, for instance, expect to see 
Richard Rodriguez (1982), who writes movingly and beautifully about his 



.< Scientific concepts which have become political notions still tend to be 
* . countered by arguments couched within the realm of scientific language. 
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upbringing and goes on record Against bill rtg.ua! education," cited 1n 

^ • r ./ ' .' ■ ; ■ •' ■ - •• •• •* . , 

Chlcano research. 

Another measurement problem is that of testing. There exi-sts an entire 

literature on minority testing and I won't go into it here except to men- 

ti on- three issues in passing. Fjrst of all, there is the meaning of .testing 

which may vary from culture to culture. Cole et al . (1971) have documented 

this in detail in their work wfth Kpelles. Donald Ertckson 1(1969) who ' 

• V 
tested the Navajo children in the Rough Rock" study wrote me 1n answer to . 

a question about the testing: "In observing the students while they re- ; 

sponded to the achievement tests, I was convinced that 1n many cases we 

were were getting measurements of attitudes regarding time, competition, 

the Importance of tests .etc., much more than we were getting data oh 

what the tests purported to tap. The typical achievement test, I fear is 

a rather stupid way of testing many American Indian students" (Private 

correspondence, June 26~ 1973). This practical -technical objection is echoed 

in the .conflict perspective-oriented attafck by .Edelsky ' e^jvr (1n press:14) 

on Cummins and Swain (1979) for, of all .things, propagating a language 

deficit theory: ■' v . •. g 

When some people interpret a test question as a hostile 
demand for disclosure, while others interpret that same 
test question as a harmless demand for a performance', 
then 1t is inaccurate to claim that the two groups ex- 
perienced the same standardized situation and task.. The .._/*• 

overall consequence of rel iance on. data from such mea- 
sures and settings 1s to make "the hegemony of the 
. successful look legitimate because of their superior < 
performance on school tests" (Orasanu et al.: 1977:61). 

Edelsky et al. 's major objection to Swain and Cummins is that. they don't 

like standardized test scores and measurement research, i.e. they deny the' 

validity of such operational izati on. The neo-marxist Interpretation apart, 



testing does remain a problem in cross-cultural research. 




There is also the problem of translation and standardization of testing ^ 
Instruments. . Osgood et aT. (/975:15ff) discuss* the problems of translation, 
basically a que^1^n>f;^hoos1ng exact equivalent or culturally cor- v 
responding Items, j e. Christian Sunday corresponds with Muslim Friday. 
They go on to point out since. the purpose of standardization is to render «. 
data from divers? samples comparable, this may well mean in cross-cultural 
research the deliberate choice of different Instructions and even pro- 
cedures. Trlairand error with native informants and careful .pretesting ! 
is an obvious strategy. 

So far the discussion has concerned methodological problems. But as 

Eisenstadt points out, the construction of problems for comparative analysis 

also engenders theoretical and analytical problems. Types constructed out 

of variables Imply some assumption about the Importance of such variables, 

such as for example Cummins' CALP and BICS. u 

Such analytical problems tend to become even more important: 
in attempts to "explain" varied types of institutions, 
organizations, or pattern of behavior in terms of some 
broader conditions. In most comparative analysis such ex- 
planation aims. . .to elucidate the Conditions under which 
such varied societal types emerge and continue to exist 
and function. . .(Eisenstadt 1969:425) 

Let me Illustrate. In order to understand the behavior of immigrants, inclu- 
ding their language behavior, we have used the constructs of ethnicity, 
ethnic groups and ethnic boundaries (Barth 1969, Sehermerhorn 1970). But 1t 
may well be that ethnicity is not the best choice of construct to explain . 
the situation of the Finns in Sweden compared With that of the Other Immigrants 
I am at this point tentatively considering the situation of the Finns as an 
extension of geographic nationalism rather than one v . 



* ' .- ' ■ ' * ' . ... '' ' ' 

of ethnic boundary maintenance, a construct the serves well for the Turks 
and the Assyrians. The choice between national ism and ethnicity then is 
an analytical problem and arguing that out will constitute the topic 
\ .f-i for a future p|p?er, but briefly, that construct with its covariation of 
variables which.nwst simply can account for the most data is usually con- 
sidered the better choice. To the degree such a choice will carry ex- 
planatory power, it becomes a theoretical problem. 

Finally I must mention the kind of problems that come under the heading of 
. ethical problems in comparative research. They have been extensively . 

discussed in the literature (see e.g. Brislin et al. 1975) and I mostly want 
just to acknowledge their existence. TESOLj^as adopted a set of ethical 
guidelines for research in English as a Second Language and I think all 
academics should discuss these matters with their students. The fault- 
finding type of research that LOfgren and'Ovrihen-Birgers tarn mention is an 
. ethical problem. Research on immigrants easily enough becomes a form of 
internal colonization, and that is also an ethical problem.- Most of all 
maybe, what do the immigrants get out of all this research, not in vague 
terms of future policies but right now? It is a glib and frequent question 
but difficult to answer. 

Sound generalization and prediction depend ultimately on good judgment, 
Which in turn depends on "an understanding of the various forces which 
underlie the process. Gaining such an understanding is more a matter of 
saturation in the situation than of scientific technique 1 ' (Simon 1969:357). 
My hope is that this discussion will help contribute to our good judgment 
in understandingr^he problems in the comparative analysis of BE so that we 
Safely and accurately generalize from the appropriate data. 

eric -':\.': * ; .- ; v;.,' 13 '.• / - \:":\'r-^'."\/ 
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